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This paper describes the results of experiments conducted with human line pilots and a cognitive pilot model during interaction with 
a new 4D Flight Management System (FMS). The aim of these experiments was to gather human pilot behavior data in order to 
calibrate the behavior of the model. Human behavior is mainly triggered by visual perception. Thus, the main aspect was to setup a 
profile of human pilots’ visual attention allocation in a cockpit environment containing the new FMS. We first performed statistical 
analyses of eye tracker data and then compared our results to common results of familiar analyses in standard cockpit 
environments. The comparison has shown a significant influence of the new system on the visual performance of human pilots. 
Further on, analyses of the pilot models' visual performance have been performed. A comparison to human pilots' visual 
performance revealed important improvement potentials 


1.0 INTRODUCTION 

The European project HUMAN (EC’s 7th 
Framework Programme) aims at developing 
virtual test pilots, in order to improve the 
human error analysis of future cockpit 
systems in early design phases, as a 
supplement of simulator tests with human 
pilots in later design phases. In HUMAN, a 
4D Flight Management System (Advanced 
Flight Management System, AFMS) and its 
user interface (Airborne Human Machine 
Interface, AHMI), developed at the German 
Aerospace Center (DLR Braunschweig), 
have been selected as systems under 
investigation. The virtual test pilots are 
instances of a cognitive architecture named 
CASCaS (Cognitive Architecture for Safety 
Critical Task Simulation, see [12]). Cognitive 
architectures, such as ACT-R (see [1]), 
SOAR (see [10]), MIDAS (see [4]) and 
CASCaS implement cognitive plausible 
theories for human perception, memory 
operations and decision making. These 
theories are independent of specific human- 
machine interfaces. Thus, cognitive 
architectures are applicable not only in the 
aviation domain, but also in the automotive 
or maritime domain. Perception of system 
and environmental states - or of entities in 
the real world in genera! - is a key factor for 
situation awareness and for decision 
making [6], The main channels for 
perception on human-machine interfaces 


are primarily eyes Tor visual perception ana 
secondarily ears for auditory perception. A 
third upcoming channel is the skin for tactile 
interfaces, but this is - to our knowledge - 
currently not implemented in any of the 
cognitive architectures mentioned before. 
Due to the importance of visual perception 
for human-machine interaction, and for 
situation awareness and decision making, 
there is a need for an accurate simulation of 
visual performance in cognitive 
architectures. 

Introduction of new user interfaces, e.g. into 
common cockpit setups, has influence on 
the visual attention allocation. Examples for 
this effect can be found in [7], This could be 
explained by the following two points: On 
the one hand, new interfaces can trigger 
attention bottom-up, meaning that the 
interface presents information in a very 
dominant way which distracts visual 
attention from other interfaces. This is often 
referred to as selective attention, where eye 
movements and shifts of attention are 
triggered by the onset of a salient stimulus 
[16], On the other hand, attention allocation 
can be affected top-down because the new 
interface provides new functionality or 
displays redundant information in a more 
accessible or usable way than other 
interfaces do. Top-down attention is caused 
primarily by underlying task models that 
comprise the allocation of visual attention. 
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Thus, a cognitive architecture that should 
simulate visual attention allocation 
humanlike requires both, valid cognitive 
theories for bottom-up attention and a valid 
task model embedding tasks on the new 
interface into the common task model for 
top-down attention. 

In this paper we will present results of 
experiments with human line pilots and a 
pilot model interacting in a cockpit 
environment containing the AHMI. Although 
the datasets have also been used to 
validate the cognitive theories for visual 
attention allocation implemented in the 
model, the main focus of this paper is the 
validation of our task model for scanning 
activities in the new cockpit setup. 

In the following section we describe top- 
down and bottom-up concepts for visual 
attention implemented in CASCaS (section 
2). Then, the experiments conducted 
(section 3) and the results of these 
experiments are presented (section 4). The 
paper closes with a short discussion 
(section 5) and conclusions (section 6). 

2.0 MODELING VISUAL ATTENTION 

Visual attention allocation is a complex 
conglomerate of top-down (active) and 
bottom-up (reactive) processes triggering 
percept actions. Top-down and bottom-up 
attention compete against each other [3], 
e.g. a salient stimulus might distract pilots 
from tasks which they are focused on. This 
is often intended, e.g. in case of warnings. 
However, a salient stimulus might go 
undetected, because top-down attention 
causes the eyes to move to an area of 
interest where the stimulus is either out of 
the visual field or absorbed by a dynamic 
neighborhood. The cognitive architecture 
CASCaS implements both processes. In the 
following subsections we will describe how 
top-down and bottom-up processes have 
been implemented. 

2.1 Top-Down Attention 

The top-down attention is driven via three 
different levels of consciousness (see 


Fig. 1), which are based on Anderson’s three 
layers of consciousness named 
autonomous layer, associative layer and 
cognitive layer [2], This is also in line with 
Rasmussen, who defined three levels of 
behavior, called skill-based, rule-based and 
knowledge-based [13], While nearly zero 
consciousness is needed on the 
autonomous layer, almost full 
consciousness is needed on the cognitive 
layer, where decision making, planning and 
problem solving are located. 



Fig. 1: The multi-layered architecture of 
CASCaS consists of components for 
perception, memory, knowledge processing 
and motor actions. 

Top-down processes on the associative 
layer are the main driving factor for visual 
attention allocation of pilots, where they 
perform well-learned rules to achieve 
specific goals. These rules describe 
normative procedures - percept and motor 
actions that match correctly specific 
situations. With regard to visual attention of 
pilots we differentiate between two types of 
procedures: (1) scanning procedures and 
(2) interaction procedures. Scanning 
procedures only contain percept actions. 
Pilots regularly perform scanning of multiple 
aircraft and environment parameters in 
order to keep situation awareness for 
current and future aircraft states. These 
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scanning activities are the main driving 
factor for visual attention in our pilot model. 
Interaction procedures contain percept and 
motor actions. They are used to interact 
with interfaces in the aircraft, such as the 
AHMI. Percept actions are needed in order 
to assess current situations and because 
we assume that pilots look at buttons before 
they press them. 


In CASCaS, normative procedures are 
described by formal rules. The rule format is 
a Goal-State-Means (GSM) format (see Fig. 
2). All rules consist of a left-hand side (LHS) 
and a right hand side (RHS). The left-hand 
side contains a goal in the Goal-Part and a 
State-Part specifying boolean conditions on 
the current state of the environment in the 
memory. Apart from the condition the State- 
Part contains memory- read operators to 
specify that, in order to evaluate a condition, 
the associated values v, of interaction 
elements ij have to be retrieved from 
memory. The right-hand side consists of a 
Means-Part containing motor and percept 
operators (writing values and reading values 
in the simulated environment), memory- 
store operators as well as a set of partially 
ordered sub-goals. 


Goal(gi) — | 

r 

^ I Memory-read(ii, v t ) | 
- 1 | Memory-read(i 2 , v 2 ) • — [ 
^Cond(v 1 > v 2 ) | 


Goal-Part 

State-Part 


Rules are connected by a goal on the left- 
hand side and goals on the right-hand side. 
This allows us to break down complex 
procedures into a hierarchical ordered 
structure, similar to hierarchical task trees 
with a small difference: Our rule-based 
description of procedures permits transitions 
from lower levels to higher levels (see Fig. 

3). Each procedure consists of 1...n goals. 
Each goal is a unique entity that is allocated 
to 1...m rules but each rule is allocated to 
exactly one goal. 



Fig. 3: Rules are connected by goals on the 
LHS and RHS 
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Fig. 2: Procedural knowledge is described is 
a specific rule format that consists a certain 
goal in a Goal-part, a State-part and a Means- 
part 


During simulation the cognitive architecture 
selects rules based on their left-hand sides 
and executes the right-hand sides. 


2.2 Bottom-Up Attention 

Bottom-up processes are unconscious and 
triggered by the perceptual component of 
CASCaS. The main driving factor for 
bottom -up attention in CASCaS is a theory 
called selective attention. Selective attention 
is an effect where salient objects, e.g. 
flashing lights, moving objects, or high 
contrasts, cause an automatic shift of 
attention towards this object [16], Attention 
shifts can also be triggered by acoustic and 
tactile stimuli, which are not investigated in 
this paper. In terms of visual stimuli, a 
salient stimulus means a discontinuity in 
space or time in the visual field. A 
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discontinuity in space represents a 
difference in a static property, like color, 
brightness, form or orientation. This could 
be for example a green dot in a set of red 
dots. In contrast to this, a discontinuity in 
time - or dynamic discontinuity - denotes a 
dynamic change, like abrupt onset, flashing 
or moving of an object. This effect may be 
restrained by the top-down process or by 
the saliency of other objects nearby, which 
suppress, with their own high saliency, other 
salient objects 

Bottom-up attention can trigger specific 
procedures on the associative layer, e g. in 
case of a flashing emergency light the 
attention of pilots should be shifted to the 
flashing light which is followed by execution 
of a procedure to handle the emergency. 

3.0 EXPERIMENTS 

In order to validate the visual performance 
of the model, experiments have been 
conducted with human subject pilots and 
with CASCaS in a functionally equivalent 
simulation environment. In the following 
sections, we will describe how the 
experiments with the human pilots have 
been carried out. 

3.1 Target System AHMI 

The main objective of our analysis is the 
interaction between the pilot flying (PF) and 
the AFMS. The AHMI is a graphical user 
interface supporting interaction between the 
AFMS and pilots. Both, the AFMS and the 
AHMI have been developed by the German 
Aerospace Center (DLR, Braunschweig, 
Germany). The AHMI supports graphical 
information about the current positions of 
the ego-aircraft and other aircrafts, weather 
conditions and flight routes. It provides a 
horizontal view (as shown in Fig. 4) and a 
vertical view. It supports onboard 
management of flight trajectories and 
negotiation of trajectory changes with Air 
Traffic Control (ATC) via Data Link to 
reduce voice-communication. The AHMI is a 
powerful tool for pilots, improving 
predictability of conflicts between aircraft or 


between planned routes and severe 
weather conditions. 



Fig. 4: The AHMI, a graphical user interface 
supporting interaction between AFMS and 
pilots 


3.2 Flight Simulator Setup 

The experiments have been conducted in 
the GECO (Generic Experimental Cockpit) 
simulator, which has been built and is 
maintained by the DLR in Braunschweig. 
The layout of the simulator has been 
derived from the Airbus A350 XWB aircraft. 

It is equipped with freely programmable 
wide-screen LCD displays and modern input 
devices like side sticks and a Keyboard 
Cursor Control Unit (KCCU), as used in the 
A380. The flight dynamics are derived from 
a VFW 614 (ATTAS), as used by the DLR 
as a test aircraft. The outside view is 
generated via three video projectors on a 
spherical screen with a diameter of 6 
meters, providing highly realistic outside 
view. The GECO is a fixed-based flight 
simulator equipped with a visual head 
tracker (AR-tracking), and an iView-X eye- 
tracker system from SMI. Eye- tracker data 
has been matched on specific regions 
representing areas of interest (AOI) where 
visual attention allocation should be 
analyzed. These AOIs were the following: 

• Airborne Human Machine Interface 
(AHMI) 

• Primary Flight Display (PFD) 
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• Horizontal Situation Indicator (HSI) 

• Engine Display (ENG) 

• Flight Command Unit (FCU) 

• Gears and Auto Break (GAB) 

• Outside view (Windows) 

In addition, pilot voices and all flight 
parameters have been recorded. 

3.3 Scenarios 

In order to analyze pilot behavior, 8 
scenarios have been defined, containing 
different AHMI-related tasks. The scenario 
that we refer to in this paper contained 3 
events that pilots had to handle. These 
events triggered pilots to perform re- 
planning of their current flight plan 
according to requirements sent by ATC. A 
flight plan is a list of waypoints the aircraft 
has to fly over or fly by. The scenario was 
divided into three phases: cruise, approach 
and landing. Communication between pilots 
and ATC has been restricted to non- 
auditory communication via the AHMI which 
allowed uplinks or downlinks of flight plans. 

3.4 Participants 

The experiments have been conducted with 
13 male and 2 female German line pilots 
recruited from German airlines. None of the 
pilots has been experienced in the usage of 
the AHMI, and only some have been in the 
GECO before. All subjects participated as 
the pilot flying (PF). The crew was 
completed by a scripted pilot, who acted as 
a pilot monitoring (PM). Scripted PMs were 
a male DLR test pilot or a female first officer 
from Lufthansa. In addition to the normal 
duties of the PM, the scripted pilot was 
responsible for the training and supported 
the debriefing and analysis by taking notes 
during the flight. 

3.5 Procedure 

The experiments were distributed over two 
days. The first day started with a general 
briefing on the project. Afterwards training 
on the AHMI and the GECO has been 


performed by the PM. After the pilots felt 
familiar with the tasks and the simulator, a 
talk-through was performed, in order to 
verify that the procedures where well- 
trained. After the talk-through was 
performed successfully, the subjects started 
to fly the first scenario. Typically, 2 
scenarios were finished on the first day and 
5 to 6 scenarios on the second day. 

4.0 RESULTS 

In this section we present results of 
analyses regarding top-down visual 
performance of human pilots and of our pilot 
model in a cockpit setup containing the 
AHMI. The analysis is based (1) on eye- 
tracker data, which have been recorded 
during the experiments with human pilots 
and (2) on log files for the pilot model. The 
output of both data sources has been pre- 
processed into a comparable format 
containing timestamps and AOIs ao/',... m 
describing where pilots have looked at a 
specific time. Each t, in the datasets is 
associated with exactly one ao/}. The 
experimental cockpit has been divided into 
7 AOIs (see section 3.2) in order to analyze 
the gaze distribution. However, the results 
presented in this paper focus on 4 AOIs 
(AHMI, PFD, HSI, windows) which have 
been selected after a first review of the data 
for the following reasons: AHMI, PFD and 
HSI are the main displays for monitoring 
aircraft and environmental states in our 
scenarios during all flight phases. The 
windows are very important for perception 
of the outside world during the landing. We 
segmented the datasets according to 3 flight 
phases (cruise, approach, landing) and 
calculated the percent dwell times (PDT) for 
each phase, respectively. PDT is a format 
representing the dwell time spent on a 
specific AOI in relation to the sum of dwell 
time spent on all AOIs observed in (%). We 
analyzed the PDTs on two levels: First, we 
performed a separate comparison of the 
results of each phase for the human pilots 
and for the model. Second, a comparison of 
human data to model data has been 
performed. 
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4.1 Human Performance 

The gaze distribution of pilots during flight 
can be seen as the main indicator of how 
important specific areas are for flying an 
aircraft - from a pilot’s point of view. 

Huettig, Anders and Tautz [9] revealed the 
dominance of the PFD in modern glass 
cockpits with a value of around 40%. For 
the HSI a value of around 20% has been 
measured. This resuit is in line with results 
published by Mumaw, Sarter and Wickens 
(see [1 1] and [14]), who analyzed the 
monitoring behavior of pilots on an 
automated flight deck. They measured 35% 
on the PFD and 25% on the HSI. Futher on, 
eye movement analyses with a Boeing 747- 
400 desktop simulator have been conducted 
by Dietz et al. (see [5]). 

In order to get an overview of our results, 
Table 1 depicts the average PDTs of our 
human subjects for each flight phase. 

Values do not sum up to 100 because 
dwells on other AOIs are still taken into 
account but are not displayed. 



Cruise 

Approach 

Landing 

AHMI 

60 

42 

21 

PFD 

15 

28 

40 

HSI 

7 

11 

12 

Windows 

6 

7 

17 


Table 1: Aggregated PDTs of human 

pilots during flight phases cruise, approach 
and landing 


In contrast to results mentioned above, our 
results reveal a dominance of the new 
introduced AHMI with a value of 60% during 
cruise phase. The PFD, with a value of 
15%, is far behind the AHMI. This 
emphasizes the role of the AHMI in our 
scenarios. HSI, with a value of 7%, is 
behind the PFD, which is in line with results 
reported in literature. During cruise outside 
view is not important, thus, with a value of 
6%, windows are behind the HSI. From 
cruise to approach PDT on PFD increases 
by 13%, while PDT on AHMI decreases by 
18%. HSI is also increasing by 4% and 
windows by 1% From approach to landing 


PDT on PFD is increasing by 12% and PDT 
on AHMI is decreasing by 21%. PDT on HSI 
is increasing by 1% and PDT on windows is 
increasing by 10%. Thus, from approach to 
landing the rank orders of AHMI and PFD 
change as well as the orders of HSI and 
windows. We assume that changes in gaze 
distribution between different flight phases 
are caused by different task models for 
each flight phase E g. the high values on 
windows during landing phase are caused 
by the upcoming landing task which triggers 
the pilot to monitor the runway. Low values 
on the AHMI during landing phase are 
caused by degradation of the navigation 
task. These changes are caused by top- 
down attention as described in section 2.1. 

4.2 Model Performance 

Results of model performance show, with a 
value of 65%, a strong dominance of the 
AHMI during cruise phase. With a value of 
31%, the PFD is behind the AHMI. From 
cruise to approach there is only a small 
change to 66% on the AHMI. PDT on PFD 
does not change. From approach to landing 
rank orders of AHMI and PFD change. PDT 
on AHMI decreases from 66% to 35% and 
PDT on PFD increases from 31% to 53%. 
HSI and windows are at a very low level 
between 0% and 2% during all phases. All 
results are presented in Table 2. 



Cruise 

Approach 

Landing 

AHMI 

65 

66 

35 

PFD 

31 

31 

53 

HSI 

0 

2 

2 

Windows 

1 

1 

1 


Table 2: Aggregated PDTs of pilot 

model during flight phases cruise, approach 
and landing 


4.3 Model Validation 

Human performance data has been used to 
validate the visual performance of the pilot 
model based on two dimensions, trend and 
local fitness, that are often used in the 
domain of cognitive model validation. 
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4.3.1 Measure of Trend 

A trend describes how a dependent variable 
v d develops in relation to an independent 
variable V;. We have measured the variable 
gaze distribution {= v d ) in relation to the 
flight phases (= v,) for the human pilots and 
for the pilot model. An aspect of model 
validity is trend consistency, meaning that 
the relation between v d and v, is the same 
for the model and for the real world aspect 
observed. In the area of cognitive mode! 
validation, the use of Pearson's correlation 
coefficient (rand r 2 ) is a common measure 
of trend (see e.g, [7] and [15]). Having a 
look at the performance of the pilot model 
applying our scanning procedure, it can be 
seen that it fits the human visual 
performance rather well with r 2 = 0.85. 

Figure 5 visualizes trends based on PDTs 
measured for the human pilots and for the 
pilot model during the flight phases cruise, 
approach and landing. 
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Fig. 5: Comparison of gaze distribution for 
human pilots and pilot model across flight 
phases 

AHMI and PFD are the most dominant 
displays during all phases for human pilots 
and for the pilot model. AHMI and PFD 
change their ranks order from approach to 
landing phase. The human data trends for 
AHMI and PFD between cruise phase 
(AHMI = 65%; PFD = 31%) and approach 
phase (AHMI = 66%; PFD = 31%) have not 
been captured for the model. Indeed, the 


trend for AHMI between these phases is 
slightly contrary to the human findings. The 
human data trend on HSI has been well 
captured for the model, where PDT is 
increasing from cruise (= 0%) to approach 
(= 2%) and then holding the level from 
approach to landing {= 2%), The model’s 
PDTs for the windows are linear for all flight 
phases (= 1%). We had problems modeling 
this AOI, because dynamic AOIs, such as a 
runway “moving” on the windows, currently 
cannot be modeled within the architecture. 
Thus, we are not able to provide the model 
with information that is gathered by human 
pilots when they are looking out of the 
windows. Nevertheless, during our 
experiments we implemented some kind of 
“blind scanning” on the windows in order to 
simulate transitions between windows and 
displays. The intention was to model the 
effect of not looking at displays (for 
whatever reason) which has been identified 
as a cause for long reaction times because 
visual signals such as flashing buttons are 
not in the visual field (see section 2.2). This 
may also impact pilots’ situation awareness. 

4.3.2 Measure of Location 

We analyzed the local fitness of gaze 
distribution by comparing the Root Mean 
Squared Successive Differences (RMSSD) 
values of human pilots and the pilot model 
as presented in [14]. Local fitness measures 
of model to human data are a bit 
problematic as trying to optimize local 
parameters bears the danger of overfitting 
the model. Instead of fitting the model to a 
static parameter value, it is more 
reasonable to fit the model into a range of 
parameter values. RMSSD can be used to 
gain insight into the differences of 
performance between an individual subject 
s, and a group g s of individual subjects s, .. n . 
We calculated RMSSD for each of the 
human subjects, pulling them one at a time, 
without replacement, from the group. In our 
case the group contained 10 subject 
datasets and we tested the fit of s-, to s 2 ...io , 
then data from s 2 to s f , s 3 .,.? 0 and so on. The 
results of these measures were 10 values, 
one for each pilot, describing the deviation 
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from the performance of the group. This 
approach has been extensively described in 
[8], We also calculated RMSSD for our pilot 
model by comparing the model dataset to 
the group of all human subjects s^.jo. Next, 
we will focus on results regarding the cruise 
phase as this is the most important phase 
for pilot interaction with the AHMI. Results 
are depicted in Fig, 6, RMSSD values for 
human subjects range from 5.52 for subject 
PF_03 to 24.11 for subject PF_09. The 
RMSSD for the pilot model is 19.21 which is 
within the range of human subject values. 
However, a comparison of this value to the 
median of human pilots' RMSSDs (= 7,63) 
shows that the model result is closer to the 
maximum than to median. 

PF_01 

PF_02 

PF_03 

PFJ)4 

» PF 05 
t; 

jf, PF_06 
" PF_07 
PF_08 
PF_09 
PF10 
Model 

Fig. 6: Comparison of RMSSD values for 
PDTs of human pilots' and pilot model's gaze 
distribution in cruise phase 

Except for subjects PF_02 and PF_09 all 
pilots are below a value of 12,0 which 
shows that these results are outliers in the 
sample. Analysis of outlier datasets showed 
that the deviations are caused by 
differences in PDTs on the AHMI. We have 
measured 60% mean PDT and 61% median 
PDT on the AHMI which is a hint on a well- 
balanced distribution. For PF_02 we have 
measured 80% PDT (= max) and for PF_09 
we have measured 39% PDT (= min). PDTs 
of PF_02 on other AO Is were much lower, 
those of PF_09 much higher respectively. 

An explanation could be that PF_02 used 
redundant information shown on the AHMI 


(such as speed, altitude) for monitoring. 
Thus, he has implemented the AHMI in his 
scanning procedure (top-down attention). 

On the other hand, PF_09 used the AHMI 
only if he had to react to ATC uplinks 
(bottom-up attention) instead of including 
the AHMI into his scanning procedure. 

5.0 DISCUSSION 

Analysis of visual attention is a useful 
means for assessment of situation 
awareness and derivation of task models for 
scanning activities in cockpit environments 
We have modeled scanning procedures for 
an advanced cockpit environment and 
performed experiments with a pilot model 
applying this procedure and with human 
subject pilots. We used the visual 
performance data recorded for the human 
pilots and for the pilot model to validate the 
visual performance of the model. While 
Pearson's rand t 2 are useful trend 
measures, RMSSD can be used to measure 
the local match between model and human 
data. Good results for Pearson’s rand r 2 are 
not sufficient to validate a model. A valid 
model must also perform within the natural 
range measured for the variable under 
observation of the human subjects. 
Comparing our result for the trend measure 
between human pilots’ and model gaze 
distribution with the results of local fitness, 
we derive the following: As the trend 
measure between model and human 
performance revealed good fitness, we 
assume that we have a rather good 
assumption of how important specific AOIs 
are for the pilots relatively to the flight 
phases. As the gaze distribution is a good 
indicator for the correctness of the scanning 
tasks in the different flight phases, we also 
assume that we have a correct 
understanding of the importance of specific 
scanning tasks performed in these fiight 
phases. However, RMSSD revealed that the 
performance of the model is at the upper 
bound of human subjects’ performance. 

This can be improved by decreasing gaze 
on AHMI and PFD, and increasing gaze at 
least for the HSI, which has not been 
modeled sufficiently. Gaze on the windows 
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has not been modeled adequately. It has to 
be discussed if it is reasonable to put 
attention on an area, whose functionality 
cannot be simulated, only to provoke effects 
related to bottom-up attention. Alternatively, 
only flight in cruise phase could be 
modeled, which has shown to be the most 
relevant flight phase for AH Ml interaction. 

6.0 CONCLUSIONS 

In this paper we have presented results 
concerning the visual attention allocation of 
human pilots and of a pilot model in an 
advanced cockpit environment. We have 
been able to show that the AHMI, a new 
interface for aircraft navigation, has a strong 
influence on the gaze distribution of pilots 
due to task models underlying the flight 
phases. Tasks (especially scanning 
activities) have been modeled in a rule 
based language. These rules have been 
applied by our pilot model as procedural 
knowledge during the flight phases. 

Analyses of human pilot performance and 
model performance in the dimensions of 
trend and local fitness revealed that there is 
still some potential left for improving the 
scanning behavior of the model. An open 
question is if it is useful to model “blind 
scanning” on AOIs whose functionality 
cannot be simulated in order to provoke 
effects related to bottom-up attention. 
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Motivation 




Support evaluation of system designs in early phases of system development process 
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The Role of Visual Perception 
for Modeling Pilot Behavior 


1 . Evaluation of situation depends on 
parameter values 

2. Parameters have to be percepted from 
the environment 

3. Pilots percept environment mainly via 
visual channel 

4. No valid model of visual perception -> 
No valid model of pilot behavior 
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Visual Attention: Top-Down 


• Active scanning behavior 

• Depends on context of situation 

* Abstraction of SEEV Model (Saliency, Expectancy, Effort, Value) 

* Probability value for each AOI 



Cruise I Landing 
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Visual Attention: Top-Down 
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Improved Top-Down Attention 


• Pilots tend to optimize scanning behavior 

• Probabilities on transitions between AOIs 

• Different probability values for each transition 
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Visual Attention: Bottom-Up 


• Reactive scanning behavior 

• Depends on saliency of objects in visual field 

• SEEV Model Saliency 
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Visual Attention Model 


• Consists of 

- Top-down attention (active) 

- Bottom-up attention (reactive) 

• Visual attention is mainly influenced by top-down 
attention 

- Considers context of different situations 

- Supports modeling of human optimization strategies 


^jImodsim world 
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Experimental Setup 

• Scenario duration: ~35 minutes 

• 15 Airline pilots 

- 13 male, 2 female 

- Average Age: 34.0 (SD 5.9) 

• Events triggering re-planning on AHMI 

• Three flight phases: 
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Target System: AHMI 


• Airborne Human Machine Interface 

* Data link communication between Crew and ATC 

* Negotiation of 4D flight plans and trajectories 

• View on ego-aircraft 

- Horizontal 

- Vertical 
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Simulator Layout and AOIs 
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Results 
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• Focus of pilots’ attention is mainly on AHMI 

* AHMI has no influence on rank order of standard 
displays 


Comparison of mean PDTs of human pilots to literature 
(aggregated for all flight phases) 
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Measure of Trend 



Cruise Approach Landing 

Flight Phases 


— Human — Model ■ AHMI #PFD A HSI AWindows 


728 



MO DSIM WORLD 

Measure of Location: RMSSD* 


1. Calculate individual 
difference of each 
human pilot to the group 

2. Calculate difference of 
virtual pilot to the group 



3. See if model is in range 
of human performance 


*in cruise phase 
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Questions/Comments 


Thank you for listening! 
Any questions? 
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